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Executive Summary 

An Agency team collaboration pilot was conducted from July 2002 until June 2003 and then extended 
for an additional year. The objective of the pilot was to test a set of tools and supporting processes to be 
considered as a team collaboration service for the Agency. In an effort to share knowledge and 
experiences, the lessons that have been learned thus far are documented in this report. Overall, the pilot 
has been successful. An entire system has been piloted: tools, adoption, and support. The pilot consisted 
of two collaboration tools, a virtual team meeting capability and a team space. For the virtual meeting 
capability, we used WebEx Meeting Center application (WebEx Communications, Inc., San Jose, CA). 
For the team space capability, we used the eRoom Hosted Enterprise Service application (eRoom 
Technology, Inc., Cambridge, MA). 1 Of the two tools that were evaluated, the team meeting tool has 
been more widely accepted. Though the team space (eRoom) tool has been met with a lesser degree 
of acceptance, the need for such a tool in the NASA environment has been evidenced. Both adoption 
techniques and support were carefully developed and implemented in a way that has been well received 
by the pilot participant community. 

The pilot administration team members have had much experience conducting similar pilots. Because 
of this experience, several of the lessons reported in this document were not newly learned but simply 
reconfirmed. More than a year into the pilot, we have also learned many new lessons. While 
they are captured in detail in this report, the major ones fall into three categories: (1) Collaborative tools 
enable NASA virtual teams to work more efficiently, (2) Adoption (change management) planning and 
implementation are critical to the successful use of the tools, and (3) Issues unrelated to the technology 
or its acceptance have significant impact on the pilot. The top 10 lessons learned from the pilot are 
listed below. 


Collaborative Tools Enable NASA Virtual Teams 

(1) A virtual meeting solution is quickly accepted by teams and provides immediate value. The fact 
that this type of application fits existing meeting norms most likely contributed to its high acceptance. 

(2) A team space enables virtual team processes to be completed much more efficiently than sole 
reliance on typical mechanisms such as telephone and e-mail. However, team spaces had a lower 
acceptance rate than the virtual meeting solution. This can be attributed to the significantly larger time 
investment and change of work habits the team must make at the onset of utilizing this type of tool. 


1 Application’s company and name at time of pilot. Presently eRoom Enterprise from Documentum (Pleasanton, CA). 
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(3) Web-based tools provide the accessibility needed for collaboration amongst NASA centers and 
their external partners. Barriers such as client software, network configurations, or specific hardware 
requirements can prohibit potential collaborators from fully participating thereby minimizing true 
collaboration. 

(4) Application service providers (ASPs) provide excellent operational capability as well as 
security — even though team members may perceive security issues with the use of ASPs since data may 
reside outside of the NASA domain. 


Adoption Planning and Implementation 

(5) The team coordinator (TC) role is crucial in the adoption process of tools, especially a team space. 
The TC should be IT (information technology) savvy and must be engaged so that he or she can serve as a 
team champion as well as an early adopter. This allows the person to motivate and support the team. Pilot 
participants with team vision, goals, and deliverables have greater probability of engagement. 

When these items are known, work processes and views of how the tools enable those processes are 
much clearer. 

(6) A team space needs to address a “pain point” that the team is experiencing. Perhaps it is a process 
that is not as efficient as could be. An example of that might be assuring that team members are reviewing 
the same version of a document. 

(7) The pilot administrators must be engaged with the pilot participants. Various mechanisms 
can be used to cultivate the exchange of information between the two groups. Proactive facilitation, 
communication, and customer-focused support are all methods of supporting the administrator-participant 
relationship. 


Pilot Issues 

(8) Executive and management sponsorship and support must exist. Workers tend to adopt tools and 
processes utilized by management. Management support also creates a higher level of visibility of the 
new way of doing business. 

(9) Teams are reluctant to heavily invest in a tool or process that will exist for a short duration 
(1 year), particularly if there is a large initial time investment as is the case with a virtual team space. 

(10) The limited scope (and funding) of a pilot results in limiting the community of people with 
access to tools (especially where tools require per-seat licensing). This impedes the adoption of those 
tools. 


Pilot Overview 

Shortly after the events of September 1 1 , 200 1 , the NASA Administrator asked the NASA Chief 
Information Officer (CIO) to improve the ability for NASA teams to accomplish work with less reliance 
on travel. NASA has many teams that are distributed across NASA centers and external partner locations 
There is currently no NASA standard method for basic collaboration beyond teleconferences and e-mail. 
However, commercial tools exist for increasing the richness of virtual team meetings, supporting virtual 
team processes, and sharing information. Best-practice companies have realized coiporate value by 
deploying and adopting these tools across the organization. It was felt that NASA would benefit by 
establishing a common language and standard methodology for collaboration. 
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Purpose 


Extensive collaboration amongst NASA personnel was inhibited since NASA has many teams that 
are geographically distributed across NASA centers and external partners. While it has always been 
somewhat difficult to accomplish work without travel, the events of September 11, 2001, and additional 
travel restrictions that followed have made it even more challenging. It has become more evident that 
NASA needs a corporate capability for its “virtual teams,” and it was strongly felt that successful 
deployment of team collaboration tools could make a difference at NASA. In addition, both electronic 
Government (eGOV) and OneNASA objectives included improving collaboration via information 
technology. The goals of the team collaboration pilot were 

(1) To assess the value of commercial collaboration tools applied to NASA teams 

(2) To refine NASA requirements for team collaboration tools 

(3) To provide a valued service to pilot participants and exceed their expectations by emphasizing 
facilitation change and adopting new work processes 

(4) To serve as a first step towards a corporate capability for team collaboration at NASA 

Business drivers and functional requirements for NASA team collaboration had been identified over the 
preceding years and updated after discussions with a sampling of potential users of this technology. This 
information then served as the basis for the selection of the tools. It was determined that a team 
collaboration capability should 

(1) Reduce reliance on travel to get work done 

(2) Enable collaboration between NASA centers 

(3) Enable collaboration between NASA and its external partners (industry, academia, other 
government agencies, and international partners) 

(4) Be easy to learn and use (usability) 

(5) Be operational and deployable in a short (1- to 2-month) time frame 

(6) Appeal to a broad-based audience (scientists, engineers, technicians, managers, and 
administrators) 

(7) Be secure — all users shall be authenticated and all communication shall be private 

(8) Have high availability and performance 

(9) Have a reasonable cost of implementation 


Process 

The process of conducting the pilot consisted of (1) recruitment and selection of pilot teams, 

(2) selection and deployment of collaborative technology, (3) development and application of team 
adoption strategies, and (4) measurement of results. 

Recruitment and selection of pilot teams . — The pilot was planned for a population of about 
500 people, and participation would be based on selecting a set of virtual teams representing a cross 
section of NASA project and work areas. As we considered recruitment and selection of pilot teams, 
we first defined criteria we felt were essential for a team to be successful in the pilot. Specifically, 

(1) The team must be geographically distributed and actively working on team tasks. 

(2) The team and team leader must be willing to adopt new work methods and ideally be somewhat 
frustrated with current distributed work methods. 

(3) The team must have a leader who is committed to actively leading the adoption of new work 
methods and tools. 
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(4) The team must name one of its members as a pilot focal point who will be responsible facilitating 
the setup and adoption of tools. 

(5) The team should be no larger than 25 people in size. 

Next, the NASA CIO sent a call for participation memo to the Agency inviting all centers and 
Enterprises to submit candidate teams for pilot participation. A selection process was performed to select 
teams that represented NASA mission-oriented and administrative areas. To account for the pilot capacity 
of approximately 500 seats, 25 teams were selected for the pilot. 

Selection and deployment of collaborative technology . — One of the main objectives of the pilot was 
to sell the concept of using collaborative technology to enable distributed teams, more so than a particular 
product or service. Thus, we did need to select collaborative applications for the pilot. This selection was 
guided by the NASA business drivers and functional requirements that had been identified. Key areas of 
consideration for pilot product selection included 

(1) Features : The pilot focus was to enable virtual team meetings and virtual team spaces. Data 
conferencing was the key feature intended to enable virtual team meetings. Data conferencing enables 
people to share presentations and live applications running on their computers with all meeting 
participants. The virtual team space required basic asynchronous features including file sharing, 
discussion lists, action item management, calendars, and workflow features. 

(2) Usability : The ease of use of collaborative technology is very important for NASA teams. People 
realize that if tools are difficult to use, team adoption will not likely occur. Teams are already stressed by 
lack of time and information overload. While usability can be rather subjective, it is a key requirement 
that came directly from NASA customers. 

(3) Ubiquitous access : A key business driver is to enable collaboration across NASA centers and 
external partners. Lessons learned from previous pilots reveal significant problems when special firewall 
ports must be open and/or when desktop client software (especially if it must be licensed and maintained) 
must be installed. Our vision was to allow collaborators to participate from anywhere using the Internet 
and common network and desktop configurations. No special rooms or network configurations should be 
required; tools should be accessible from any office, hotel room, telecommuting site, partner location, and 
so forth. 

(4) Security : It is vital that the privacy of NASA team information and processes be maintained. All 
team members must be authenticated and all communication must be private. All support processes must 
work in concert with the guidance from NASA’s standards for information technology security. 

(5) Multiplatform functionality : NASA (and some of its partners) has a heterogeneous computing 
environment consisting of Windows, Macintosh, Linux, and Unix workstations. 2 It is essential that 
collaborators be able to participate using their normal work environment, since this is where their work 
(applications and data) resides. 

Other issues considered include product maturity, vendor viability, training alternatives, cost, and 
deployment options. With respect to deployment options, the NASA CIO and the team at Glenn Research 
Center were interested in pursuing the option of providing the pilot infrastructure via an application 
service provider (ASP) rather than deploying in house. The ASP model is being driven by fundamental 
enabling trends in network services; specifically, the relentless increase in network bandwidth. From a 
Government perspective, the ASP model provides a provocative option for information technology (IT) 
outsourcing. ASPs can give NASA access to the identical services that are competing for customers in the 
commercial marketplace. In addition, since this was intended to be a pilot, we were interested in both a 


2 Windows is owned by Microsoft Corporation, Redmond, WA; Macintosh by Apple Computer, Inc., Cupertino, 
CA; Linux by Linus Torvalds, San Jose, CA; and Unix, by The Open Group, San Francisco, CA. 
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short deployment time and avoiding capital costs (e.g., purchasing and installing servers and software 
applications). 

Finally, with inputs from Gartner Consulting (Gartner, Inc., Stamford, CT) and the META Group, 
Inc., (Stamford, CT) and from previous piloting and standards work at Glenn, we recommended a pilot of 
two ASP-hosted capabilities: 

(1) Virtual team meetings via the WebEx Meeting Center service 

(2) Virtual team space via eRoom Hosted Enterprise Service 

To help assure a short deployment time, support processes and plans were developed and implemented. 
These included e-mail, phone, and online assistance, documents such as a pilot frequently asked questions 
(FAQ), customer support documents, and on-line help, as well as other important elements such as hours 
of operation and maintenance, priority levels and response time, and points of contact databases. 

Development and application of team adoption strategies. — Realizing it would be a significant 
challenge to transition pilot teams to new ways of working, a tool adoption plan was developed. We also set 
realistic customer expectations and planned for customer- focused technical and administrative support. 

The adoption plan focused on the following techniques: 

(1) Defining key roles : Key roles and responsibilities of pilot personnel were defined prior to the 
creation of the adoption plan. The three key roles were the team leader (TL), team coordinator (TC), and 
team facilitator (TF). A critical success factor for a team to adopt collaborative technology is that one 
team member must become an active proponent of the required change in work habits and use of tools. A 
key role then was that each team must have a defined TL who accepted the responsibility of actively 
leading the team into the ongoing application of team collaboration tools. In addition to the TL, each team 
was to name a person on the team to act as the team tool expert (or TC), who would be responsible for 
identifying opportunities to apply tools, tool setup (i.e., directory structures), and immediate assistance to 
team members. A member from the Glenn team was assigned to serve as the TF for each team. The TF 
would be responsible for developing a relationship with and checking in on the TL and TC to assist with 
the team adoption of tools. 

(2) Providing proactive facilitation : From past pilot experiences, we learned that simply giving people 
access to collaborative technology usually resulted in the tools not being effectively used, if used at all. 
We decided that active facilitation with the teams would go a long way in the adoption of the tools being 
provided. TFs would proactively monitor teams by checking in with teams on a regular basis, attending 
team meetings when appropriate, and monitoring the team space. TFs would then be in a good position to 
encourage and assist in tool adoption. 

(3) Facilitating monthly pilot forums : One way to encourage adoption is to facilitate sharing of results 
and best practices among all the teams participating in the pilot. Each month, we scheduled a virtual 
meeting with all of the team leaders and team coordinators. The forums were used to obtain feedback as 
well as share team information and best practices of tool use. The best practices shared at these forums 
included the use of WebEx to host a large international conference, the use of WebEx to collaborate on 
video content, and the use of eRoom to create a project operating plan with multiple authors. The forums 
were also used to provide technology briefings such as tool tips and features of new software versions. 

(4) Providing tool technology training : An understanding of the use and purpose of all tool features 
can facilitate adoption of tools. We planned for vendor-supplied tool training, both instructor-led as well 
as self-paced. eRoom training was completely led by an instructor and was provided at the onset of the 
pilot. Further into the pilot, it was provided on an as-needed basis by the pilot administrators. WebEx 
offers virtual instructor-led training sessions and self-paced instruction on an ongoing basis at no 
additional cost. 

Measurement of Results. — A key objective of the pilot was to assess the value of team collaboration 
tools as applied to NASA teams. To that end, we decided to administer two surveys to pilot participants — 
one before the application of team collaboration tools and another after the teams had used the pilot tools 


NASA/TM— 2005-213210 


5 



for a fairly significant period of time — about 9 months. The focus of the first survey was to both measure 
the prepilot state of each team’s ability to get distributed work done and to assess basic team collaborative 
skills (e.g., trust, communication, etc.). The focus of the second survey was to measure each teams’ 
ability to get distributed work done while using the pilot tools and to gather additional feedback on the 
pilot such as how well business drivers were met. 

In addition to the surveys, customer feedback was collected by team facilitators and at the monthly 
pilot forums. 

The survey distributed before the pilot is found in appendix A, and the survey given after the pilot is 
found in appendix B. To aid the reader, a list of acronyms used in this report is found in appendix C. 


Lessons Learned 

The pilot administrators brought to this pilot experience and knowledge from conducting past IT 
pilots. The lessons learned during those earlier pilots aided in the success of the team collaboration pilot. 
Realizing the importance of documenting those lessons, it is with forethought that we provide a summary 
of the lessons learned during this pilot. This section is divided into three major areas: (1) adoption issues, 
(2) technology issues, and (3) general pilot issues. Requirements for team collaboration are also listed. 


Adoption Issues for Team Collaboration at NASA 

Tool adoption is key to obtaining value from collaborative technology. Technology is only half the 
battle. It is difficult for one person to change habits, let alone getting 10 to 25 people to change them 
together. Executive acceptance is also critical to success. The success of any project initiative is shown to 
be directly linked to the degree of executive acceptance, and as such it is absolutely necessary that senior 
management have a comprehensive understanding of the business benefits of the system that is being 
evaluated. Upper management support, TL involvement, and forming a critical mass of team members 
with technological skills are all methods that contribute to the likelihood of tool adoption. 

Lessons were captured regarding overall tool adoption as well as each of the specific pilot applications. 

Overall . — The following were learned about adopting tools for team collaboration in general. 

Lesson: The team must be committed to working differently. The TL in particular must commit 
and actively lead the group into new ways of working. If there is not a commitment to work differently, 
real or perceived barriers to adoption grow both in number and size and seriously jeopardize the 
possibility of incorporating the tool into a team’s normal workflow. 

Lesson: Key pain points or impediments for the team must be identified. Identification of pain 
points focuses application of team collaboration tools on areas of maximum benefit. Expected results of 
utilizing the toolset must be clear and valued by the team. Valuation is directly related to the number and 
size of the impediments. Greater value is perceived when impediments are reduced. 

Lesson: A system for how team work will be automated by technology must be established. On a 

personal level, people create a system for themselves, often enabled by their personal computer, including 
folder structure, method of tracking their actions, and in some cases establishing processes. Likewise, a 
team system must be established to organize team information, actions, and processes. Each team must 
designate a coordinator (TC) to ensure that the team makes use of the tool as appropriate, to implement 
and maintain the structure for the team's information according to agreed upon criteria, and to be the 
primary team contact for issues regarding the use of the tools. Without a person who is dedicated and 
available for this function, most teams do very poorly in adoption of the tools. There needs to be 
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executive support of the TC to empower him/her to get a team to incorporate the collaborative tools into 
the team’s normal work patterns. As an example, The Payload Safety Review Panel (PSRP) successfully 
adopted WebEx in part because of a very engaged TC. The TC (1) developed a training program for all 
team members, which included PSRP and support engineers as well as payload organizations, (2) effected 
cultural change throughout the team, one person at a time, and (3) developed a process for the scheduling 
and notification of WebEx meetings. 

Lesson: Team acceptance of the new system must be established. Team members must feel vested 
in the system and actively use it for team work. They must also have an agreement among themselves for 
how they will work together using the new system. Though the entire team must buy into the new system, 
cultural change often occurs with one individual at a time. 

According to one TC’s philosophy, there are several roles as it pertains to the adoption of a new 
system: innovators, hesitators (perfectionists and procrastinators), and resisters. The innovator thrives 
with the new system as their motto is “Change can be fun!” Elesitators include both perfectionists who are 
uncomfortable with displaying their lack of knowledge and procrastinators who often proclaim that they 
are too busy for training. Resisters feel that change is bad, and often management support is needed to 
reinforce the need for change. 

Team Meetings . — The following lessons were learned about the tool applied for team meetings. 

Lesson: Conferencing (e.g., WebEx for the pilot) is quickly accepted by teams and provides 
immediate value. Meetings are a common process at NASA. Enriching the meeting experience with a 
tool like WebEx fits fairly simply into this existing process. The ability to share live content from any 
computer in a meeting significantly increases the richness of the communication. Several teams 
eliminated travel due to this increase in communication. For example, the Stennis Space Center Office of 
Technology Transfer (OTT) successfully utilized WebEx to conduct two very high level and sensitive 
meetings in place of traveling to Washington, DC. The OTT managers from all NASA centers 
participated in the meeting. A significant amount of time was saved using WebEx — 4 to 6 hours versus 
3 days. 

Note: See the Post-Pilot Survey (appendix B) for additional points. 

Virtual Team Space . — The following lessons were learned about the tool applied for team space. 

Lesson: Teams are reluctant to heavily invest in a tooEprocess that will only exist for a short 
duration (1 year). Since team space does not fit into an existing work pattern (as compared to 
conferencing), more time is needed for teams to adopt. There was reluctance to heavily invest in a tool 
that might be available for the short term. 

Lesson: Training should be provided just in time for active use of the tool. If training is offered 
before individuals are ready to use the tool, its usefulness is minimized. Training should cover the 
features of the tools, but just as importantly, training needs to demonstrate how the tool can effectively be 
incorporated into automating a team’s existing processes. 

Lesson: Piloting a team space is better accomplished by targeting an entire user segment 
(i.e., the entire project, program, or enterprise), rather than distributing seats to small teams at 
the working level. The training and needs assessment should be done with the targeted group so that 
they learn the common language of using the tool, thereby having a better chance for broader acceptance 
since everyone will be on the same page regarding how the tool is going to be incorporated into their 
work processes. 

Note: See the Post-Pilot Survey (appendix B) for additional points. 
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Technology Issues for Team Collaboration at NASA 


Technology has been an incredible enabler for distributed teams to get more work done. High-speed 
network connectivity and feature-rich applications allow virtual teams to accomplish their work remotely 
in a very efficient manner. For the technology selected as part of this pilot, particular attention was paid to 
selecting tools that supported two critical requirements for NASA: ease of use and interoperability in a 
multiplatform environment. Other areas that were very important were availability, security, and service 
delivery. While the tools generally worked as advertised, there were a few technology issues that surfaced 
during the course of performing this pilot. We felt the issues were grouped somewhat into four topic areas 
of (1) application service providers, (2) multiplatform issues: operating systems and browsers, (3) security 
issues, and (4) ubiquitous access: browsers and Internet. 

Application service providers . — The pilot was conducted using application service providers (ASPs). 
So while we piloted the tools and their adoption, we also piloted the mechanism for offering the service. 

A driving force for using ASPs was the short turnaround time needed to initially offer the services. Our 
experience using ASPs was mostly a positive one. Generally, operations and support have been excellent 
for both tools. Software updates took place with little or no disruption to the end user. Performance of the 
tools (e.g., end-user response time) was generally very good. Pilot participants were inconvenienced on a 
few occasions when there were problems with the tools. There also were some unresolved problems that 
were due to limitations with the operating systems, browser applications, and firewall restrictions in the 
systems NASA used to access the services. 

While WebEx had a couple of operational incidents, they were addressed in a timely manner by 
WebEx support staff. There were numerous software updates that occurred over the course of the pilot. 
These updates would have taken more administrative resources if the Web-conferencing service were 
offered in-house, since we would have needed to install and test in a development environment and then 
move them to a production system. It was also very helpful to have access to beta sites with the next 
release of the tool to test and evaluate any impacts before it was moved into the production environment. 
The overall availability of the service was very good as was the response time of WebEx ASP support. 

We experienced literally no down time on eRoom. However, on the occasion that we needed 
assistance, response time for eRoom ASP support was poor. 

Lesson: ASPs provide excellent operational capability as well as security. However, team members 
may perceive security issues with the use of ASPs since data may reside outside of the NASA domain. 

Lesson: ASPs allow for virtually trouble-free software upgrades and reduce need for in-house 
resources. 

Lesson: Operating systems, browser applications, and firewall restrictions can pose problems 
for ASPs in the NASA environment. 

Multiplatform issues: operating systems and browsers. — Multiplatform access is needed but is also a 
challenge. Both eRoom and WebEx support Windows, Mac OS 9/X, Linux, and some Unix variants, but 
you still find more issues and functionality gaps on the non-Windows platforms. This is somewhat hard to 
overcome in a Windows-dominated world. However, we believe we are as close to lull interoperability as 
possible considering there are many Windows-only applications that were purposely avoided. There is 
also a challenge with multiple browser types (Netscape, Internet Explorer, Mozilla, and Safari) and 
versions in use. 

Because of the platform diversity within NASA, it would be useful to know which operating systems 
and browsers are currently being used and which ones are in the queue for future deployment. Likewise, 
consideration must be given to the NASA environment when a new release of software is made available. 
As with any other software, the benefits of a release must be weighed against any complications it might 
bring to our desktop environment. So it is important that thorough testing occur before a decision is made 


NASA/TM— 2005-213210 



to deploy the software. It is also worth noting that the tool feature set may vary from one platform to 
the next. 


Lesson: Multiplatform goes beyond types of desktop (PC, Mac, Unix, etc.) and must include 
versions of operating systems and browsers. 

Security issues . — This pilot surfaced many IT security issues that had to be addressed. It was 
imperative that a security plan be developed to cover matters such as NASA Procedures and Guidelines 
(NPG) 2810 conformance and in particular the Business and Restricted Technology (BRT) data category 
that is of interest to many customers. Some customers expressed interest in collaborating on International 
Traffic in Arms/Export Administration Regulations (ITAR/EAR) data. Additionally, the service was 
provided beyond NASA proper since collaboration often extends to external partners in industry, 
academia, and other Government agencies. As a result of that, it became clear that a process must be 
developed to address external partners agreeing to the service providers’ and data owners’ rules and 
regulations on the use of a Government service. All of these security issues were managed by working 
cooperatively with the NASA Information Technology Security (ITS) personnel and the pilot vendors 
(WebEx and eRoom). 

Pilot participants stated a desire for stronger authentication. However, they also wanted an easy way 
to securely access systems, such as the RSA SecurlD (RSA Security Inc., Bedford, MA) tokens, so that 
they would not have to remember yet another username and password. 

Lesson: A balance must be struck between security and information sharing through 
collaboration. There are tradeoffs between the usability of a service and making it too restrictive. 

A tiered approach for various security levels should be developed. 

Ubiquitous access: browsers and Internet . — Access via ubiquitous infrastructure is a big win. We 
chose products that work with in-place network and desktop configurations so that no real infrastructure 
changes were required (e.g., we did not require additional firewall ports open; we used SSL, and we did 
not require any active installation of special or licensed desktop software — just quick downloads when 
accessing services). And because most teams need to interact both across NASA centers and with external 
partners, ubiquitous access was a key enabler. 

The tools operated well in a wide range of firewall and proxy server implementations and 
configurations. With WebEx, there was only one unresolved firewall and proxy server issue with one 
NASA center at the conclusion of the first year of the pilot. 

Lesson: Web-based tools provide the accessibility needed for collaboration amongst NASA 
centers and their external partners. 


Conducting Pilots 

Just as knowledge was acquired with respect to the adoption (change management) issues and 
technological issues involved in piloting the two collaborative tools, we also gained knowledge about 
conducting IT pilots. The lessons captured below will hopefully aid in conducting more efficient and 
effective pilots in the future. 

Lesson: Executive sponsorship and support must exist. 

Lesson: Develop and adhere to criteria for team selection. If a team is participating in a pilot 
simply because they were told by management “thou shall participate,” the chance of engagement is 
greatly reduced. Team members — in particular the TLs — need to see the value of adoption. 
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Lesson: Pilots need to be customer focused. Both communication and support should be just as 
people centered during a pilot as it is in a production environment. Multiple methods can be utilized to 
share information as well as to receive feedback. A pilot Web site, e-mail, surveys, a discussion area, and 
monthly forums were all used to communicate during the pilot. Monthly forums were also useful for pilot 
participants to share best practices, ideas, and concerns with fellow pilot participants. A support plan 
should include necessary elements such as pilot FAQs, on-line help, problem escalation and resolution, 
and customer support documents. 

Lesson: Customer expectations must be defined. What you expect from customers and what they 
should expect from you must be clear. This includes the duration of pilot, service levels, and so forth. 

Lesson: Pilot participants with team vision, goals, and deliverables have greater probability of 
engagement. Knowing one’s goals and/or objectives is sometime difficult because of fairly broad Agency 
shifts. Also, participating in a pilot is often viewed as a layered add-on. Therefore engagement in the pilot 
can be superficial. 

Lesson: Plan beyond a successful pilot. Be prepared for a larger scoped service if the pilot is 
successful. This includes procurement, infrastructure, security, and other areas. 


Key Requirements for Team Collaboration at NASA 

One of the first steps in defining the pilot was to draft business requirements (or drivers) for a team 
collaboration service at NASA and verify them with NASA focus groups. Those requirements have been 
further verified by our pilot participants. Key requirements have been developed. Some critical 
requirements are listed below. The team collaboration service must 

(1) Enable NASA teams to participate from any NASA desktop computer 

(2) Enable external partner participation without requiring reconfiguration of existing network 
perimeters 

(3) Be easy to learn and use 

(4) Be based on a leading commercially available Internet service 

(5) Conform to NASA NPG 2810 IT security guidelines 

(6) Provide multiplatform support (Windows, Macintosh, Linux, and Unix) in combination with 
NASA-recommended Web browser applications 


Survey Results 

Surveys were conducted as part of the pilot to assess the value of team collaboration tools. There 
were two major surveys conducted during the pilot: a Pre-Pilot Readiness Assessment (appendix A) 
conducted at the start of the pilot and a Post-Pilot Survey (appendix B) conducted 9 months into the pilot. 

Both surveys measured the ability to get distributed work done. The Pre-Pilot Readiness Assessment 
was focused on evaluating the work environment and benefits of collaborative work. Issues such as trust 
and motivation were queried. The Post-Pilot Survey was more focused on the use of the pilot tools. As a 
result, only a limited cross analysis between the two has been done. Another limitation that prevented 
greater analysis of the two sets of data was the different audiences for the two surveys because a number 
of teams did not participate for the duration of the pilot. There were several reasons why teams did not 
participate for the duration: (1) Projects and programs were cancelled; therefore teams were disbanded. 
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(2) A team’s work processes were u nk nown or did not lend themselves to virtual collaboration, and 

(3) The abundance of team responsibilities did not allow adequate time to learn and incorporate new tools 
into their work processes. In addition, other teams entered the pilot midyear through the pilot. Yet another 
difference in the survey audience was the first survey was distributed to all known pilot participants 
whereas the second survey was distributed only to team leads and coordinators. 

The Pre-Pilot Readiness Assessment consisted of three major sections: demographic, social aspects of 
collaboration, and lastly the current state of the collaborative tool environment for virtual teams. The rate 
of return for the survey was 23 percent (137 out of 593 responded). Summarizing the data returned from 
the survey: 

•Respondents represented a cross section of NASA and its external partners. In fact, representatives 
from all the NASA centers responded to the survey. 

•A high level of trust among workgroups existed, and fellow workgroup members were motivated and 
adequately engaged in the work. 

•Workgroup members openly communicated, and to a lesser degree managed knowledge. 

Complete details of the Pre-Pilot Readiness Assessment can be found in appendix A. 

The Post-Pilot Survey (found in appendix B) yielded more information about the technology and 
the implementation of the pilot. The Post-Pilot Survey had a 24 percent rate of return (126 out of 517 
responded), and again all centers were represented in the survey responses with 4 1 percent of the 
respondents being a support service contractor (SSC) or external partner. While the majority of the 
respondents used a PC-Windows platform, just under 25 percent of them used Mac or Linux platforms. 

From the results of the survey in the table below, it appears the original business drivers were met. 
Highlights from the survey include 

•Teams that had a clear mission, schedule, and deliverables more readily embraced the pilot tools and 
had greater success using them. 

•Individual responses were aligned with their respective team response (i.e., not many outliers). 

•Collocated teams that had access to WebEx still received benefit from the tool. 

•Teams with access to both eRoom and WebEx often only used one of the tools. 

•Pilot participants using eRoom were often collocated and able to meet face-to-face. This might have 
resulted in less of a need to participate in virtual meetings. 

•Pilot participants using WebEx often utilized asynchronous tools with which they already had access 
and familiarity (and not eRoom). 


Business requirement 

Applicable post-pilot survey 
question number(s) 

Requirement met? 

eRoom 

WebEx 

Tools shall reduce the reliance on travel to get work done 

3, 10 

Not 

necessarily 

Yes 

Tools shall enable collaboration between NASA centers 

4, 11 

Yes 

Yes 

Tools shall enable collaboration between NASA and its external partners 

4, 11 

Yes 

Yes 

Tools shall be easy to learn and use 

5, 12 

No 

Yes 

Tools shall appeal to a broad-based audience 
(scientists, engineers, managers, and administrators) 

NA 

Yes 

Yes 
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Concluding Remarks 


Conducting this pilot has proven to be valuable to the Agency in many ways. While sharing the 
aforementioned lessons learned is intended to broaden the knowledge within the Agency, we also feel that 
the pilot served to yield the following discoveries and confirmations: 

Increased productivity and efficiency are better proved by pilots as opposed to paper studies. Vendors 
often tout their ease of use, reliability, full feature sets, and so forth. However, it is not until customers use 
a product that these characteristics can be verified. 

Requirements can only be validated through the actual use of a product. Often an initial set of 
requirements evolve as a tool is used in one’s work processes. Requirements are updated — deleted, 
modified, and added — as the tool is used. The pilot then serves to yield a better set of requirements. 

Lastly, the experience gained during a pilot is valuable as one plans for transition to an operational state. 
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Appendix A 

Pre-Pilot Readiness Assessment 


The survey presented in this appendix was administered by the Principal Center for Workgroup 
Hardware and Software at the NASA Glenn Research Center. It was distributed to the participants of the 
team collaboration pilot to assess their proficiency in collaborative skills and practices before the pilot 
took place. 
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Pre-Pilot Readiness Assessment 

The Survey 


Collaboration is a socio-technical process that involves social processes and technology to 
achieve high levels of workgroup performance and productivity. To help workgroups better 
exploit their collaboration skills throughout the course of the pilot activity, each workgroup was 
surveyed to assess their current strengths in the four key skill areas that are considered important 
to effective collaboration in a virtual workspace. While this survey instrument is by no means a 
comprehensive cultural assessment, it does provide general insight into the workgroups' skill 
proficiency so that any necessary incremental changes can be implemented. The survey was 
conducted via the Internet using Facilitate.com. 

General data was collected from 137 (out of the 593 mailed) respondents from various Centers 
throughout the Agency. The survey questionnaire consisted of three parts: 

Part I of the questionnaire asks for user demographic information. This information will 
be used to identify the participating workgroups. This information will also help in 
defining the type of work that workgroup members are actively engaged in. 

Part II contains questions relating to the social aspects of utilizing collaborative 
workgroup technology. The questions in this section are specifically focused on the core 
skills, attitudes, and behaviors that are essential to successful collaboration in a virtual 
environment. Responses obtained in this section will give a general view of the 
workgroup’s skills for effective collaboration. 

Part III contains questions about the state of the current virtual team/collaborative tool 
environment. These questions will help assess the efficiency and effectiveness of current 
collaborative workgroup technology and will provide a baseline for documenting any 
changes in workgroup productivity. 
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Survey Limitations 

The data collection tool was not robust enough to provide adequate error checking and exception 
handling. Consequently, scores from two non-responsive respondents had to be eliminated. In 
some cases, survey respondents did not answer all of the questions. Total responses will 
therefore vary from question to question. 

A statistical ly-signi Hcant number of respondents (approximately 20%) did not select the proper 
workgroup (question 2). As a result, the survey results could not be analyzed and reported for 
each workgroup (sample population - pn). The results discussed herein are the results for the 
entire group of survey respondents (population P). This is probably the most unfortunate 
limitation of the survey. 

Survey Results 

The survey results are provided in this report, which is organized according to the survey layout. 
Part I of this report provides demographic infonnation such as Center location, workgroup name, 
and workflow type. Part II provides the results of the collaborative skill assessment. The four 
skill dimensions are trust, motivation, open communications, and knowledge management. Part 
III contains the results on the respondents' experience with their current virtual enviromnent. 
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Center Participation 


Respondent Participation by Center 
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Return Rates 


Analysis 

A 23% return rate (137/593 = 23%) provides a significant statistical sampling for this 
assessment. The return rate is slightly higher than the typically-accepted return rate for web- 
based surveys (20%). This suggests that participants care about collaboration and are willing to 
explore alternative ways to enhance and improve their collaboration skills and abilities. 


Benefits 

Benefits Respondents Would Like To Receive Through Pilot Participation 


Stated Benefit 

Frequency 

Make group information more accessible 

115/137 = 84% 

Provide a virtual context for meetings 

98/ 137 = 71% 

Provide a means for problem solving 

93/137 = 68% 

Identify project issues 

89/137 = 65% 

Encourage all members to contribute 

80/ 137= 58% 

Enhance team building 

76/137 = 55% 

Maintain historical project records 

74/ 137= 54% 

Make lessons-leamed more accessible 

71/137 = 52% 

Increased creativity 

66/137 = 48% 


Current Workflow 


Workflow 


Reciprocal 
Work Flow 
27 % 



Independent 
Work Flow 
26 % 


Sequential 
Work Flow 
14 % 


Team 
Work Flow 
33 % 
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Analysis 

Survey respondents spend slightly over one-third of their time working in a team environment 
where they collaborate to solve problems and share information to fulfill the charter of the group. 
In this arrangement, ah share in the responsibility of the work. Respondents spend slightly more 
than one-quarter of their time in reciprocal work arrangements where the work and activities are 
exchanged back and forth. In this work flow format, responsibility for the work may shift as the 
work is passed among group members. Respondents spend one-quarter of their time working 
independently. Respondents spend the remaining time in an arrangement where work and 
activities flow between group members in one direction. 


Part II - Collaborative Skills Results: All Workgroups 

Introduction 

Research on virtual teams and workgroup collaboration shows that there is a connection between 
collaboration and workgroup culture. This study specifically examines the social aspects of 
organizational culture that have a significant impact on workgroup collaboration. Drawing upon 
the literature, four key competencies are identified and assessed, namely, trust, open 
communications, motivation (engagement), and managing knowledge. 

Central to this research is the premise that workgroups with high competencies in these major 
collaborative skills are most likely to succeed in transferring these competencies to a virtual 
environment using virtual team collaboration technology. See the white paper entitled. The 
Affects of Workgroup Culture on Collaborative Tool Adoption, for a comprehensive discussion 
of this research. 

Overall mean average scores for each skill category are based on a Likert scale where 1 = 
Strongly Disagree and 5 = Strongly Agree 

Trust 

Question: I trust my fellow workgroup members: 


Trust 

Category Description 

Frequency of 
Responses Over 3 

Are competent in performing their jobs 

123/134 = 92% 

Respect authority when appropriate 

122/134 = 91% 

Will generally keep the promises they make 

119/134 = 89% 

Feel free to contribute to the success of the group 

119/134 = 89% 

Express their true feelings about important issues 

113/134 = 84% 

Have a high level of trust and respect for each other 

111/134 = 83% 

Collaborate on projects and tasks even when it takes time away from other 
duties 

92/133 = 69% 

Will openly acknowledge their own mistakes 

83/134 = 62% 


Overall Mean Average Score: 4.06 
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Analysis 

The results reveal that the respondents collectively hold a high level of trust among members 
within their respective workgroups. Respondents trust that workgroup members: 

• Are competent in performing their jobs, 

• Respect authority when appropriate, 

• Will generally keep the promises they make, 

• Feel free to contribute to the success of the group, 

• Express their true feelings about important issues, and 

• Have a high level of trust and respect for each other. 

Respondents reported slightly lower perceptions of trust in workgroup members' willingness to 
collaborate on projects and tasks when it takes time away from other tasks and duties. 
Respondents also reported slightly lower perceptions of trust that workgroup members will 
openly acknowledge their own mistakes. In a collaborative environment, individual needs must 
often compete with the needs of the workgroup. In most cases, individuals will protect their 
personal interests unless they realize a greater benefit in doing otherwise. For these reasons, it is 
not surprising to find a reluctance to trust that workgroup members will subordinate their 
individual interests to the overall needs of the group. Nonetheless, it is important to note that 
high trust scores were not anticipated. 

High levels of trust increase the likelihood that workgroups will experience and realize the full 
benefits of team collaboration. These high scores suggest that workgroup members are willing to 
trust the information they share because they trust the competency and integrity of each other. 

These scores also suggest that despite the fact that Centers often ‘compete’ for the same program 
dollars and other resources, a high level of trust and respect still exists amongst team members. 
While spirited competition among Centers may exist, team members are willing to work 
collaboratively and cooperatively for the good of the whole team. When a program is operating 
across Center boundaries, workgroup members still want to succeed, for themselves and the 
program. 
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Motivated/Adequately Engaged 


Question: I am confident that my fellow workgroup members: 


Motivated/ Adequately-Engaged 

Category Description 

Frequency of 
Responses Over 3 

Are generally motivated to succeed 

123/132 = 93% 

Are involved in making appropriate decisions that affect their work 

117/132 = 88% 

Generally have a sense of belonging to the workgroup 

110/132 = 83% 

Consider the success of the workgroup as their success 

108/132 = 82% 

Understand the overall mission and strategy of the workgroup and how 
their work supports it 

103/131 =79% 

Feel adequately- engaged in shaping the outcomes and deliverables of the 
workgroup 

103132 = 78% 

Clearly understand their role in, and responsibilities to, the workgroup's 
effort 

96/132 = 73% 

Understand how the outcomes and deliverables of the workgroup are tied 
to the business strategy and objectives of the organization 

83/131 = 63% 


Overall Mean Average Score: 4.00 
Analysis 

The results suggest that the respondents collectively believe that workgroup members are highly- 
motivated to succeed and take ownership for the overall success of the workgroup. Respondents 
are involved in making decisions that affect their work and feel a sense of ownership for their 
products and deliverables. 

Respondents have a sense of belonging to the workgroup and this suggests that there is a high 
degree of comfort with the identity of the workgroup. It is not unlikely that this strong sense of 
belonging is a motivating factor in workgroup members' willingness to contribute their best 
effort and take ownership for the success of the group. 

While respondents generally understand the overall mission and strategy of the workgroup, 
nearly 30% indicated that they were unclear on their individual role and responsibility to the 
workgroup’s efforts. Moreover, nearly 40% indicated they were unclear on how outcomes and 
deliverables are tied to the overall business strategy and objectives of the organization. 

Clarity of purpose is among the best predictor’s of team success. Role clarity is important to 
collaboration activities because individuals will put forth their best effort when they understand what 
they are expected to do and why they were selected to do it (Santous, 1993) (Fullan, 2001). 
Individuals will have an intrin s ic caring about their tasks and embrace the importance of their cause. 

Individual workgroups participating in this study would do well to take the extra time to map the 
current roles of each workgroup member to the charter and then to the mission. This activity will 
help clarify how these individuals contribute to the mission and business of the overall 
organization they serve. 
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Open Communications 


Question: My fellow workgroup members: 


Open Communications 

Category Description 

Frequency of Responses Over 3 

Pass on new or useful information as appropriate 

111/131 = 85% 

Actively listen to the thoughts, feelings, and ideas of others 

108/131 = 82% 

Provide timely information about changes in current plans 

96/132 = 73% 

Have easy, rapid access to key information 

83/131 = 63% 


Overall Mean Average Score: 3.88 
Analysis 

Respondents report that they experience meaningful communication as workgroup members pass 
on useful information and actively listen to the thoughts, feelings, and ideas. Nearly 30% of the 
respondents indicate that they are not informed about changes to current plans in a timely 
manner. This is a common dilemma of distributed workgroups and, in most cases, can be 
addressed by providing more frequent meetings, either face-to-face or virtual. Another way to 
increase communication is the use of web-based news sources like eRoom, Livelink, and similar 
tools that have event-based email notification and places project information in a single location. 
This type of technology will only work if the project members are disciplined enough to create a 
record of the changes. Nonetheless, workgroups would benefit from identifying a method for 
communicating project information as well as a communication plan for ensuring the timely 
release of news to all parties that will be affected by the information. 

Nearly 40% of the respondents indicate that they do not always have easy, rapid access to critical 
information. Workgroups are already intuitively aware of this dilemma as one of the primary 
benefits they hope to realize through this pilot is gaining more access to group information. 

It is nearly impossible to isolate the root causes of this dilemma without further investigation. 
However, social structure, technology limitations, and/or organizational structure may contribute 
to the problem. In many cases, knowledge and information is closely held by tightly knit group 
or subgroups, making it nearly impossible to obtain unless members sanction its release. Cultural 
norms and organizational design can also impact accessibility to information - it may not be an 
acceptable practice to cross organizational lines to access information. This may be particularly 
true for workgroups that function in a hierarchical organization. Also, employees may not be 
adequately compensated to share information, since their perfonnance evaluations are based on 
individual, not team performance. 

While technology such as email and document management tools play a major role in storing 
and exchanging information, other tools may be needed to harvest the full potential of that 
information. Tools such as Expert Locator and Tacit Knowledgemail which identify likely 
experts based on topics within a common directory or based on email content. 
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A legitimate reason for inaccessibility to key infonnation may be poor assimilation of 
knowledgeable members into the workgroup. These individuals are not sure what type of 
information to share and how to share what they know, especially if communication processes 
are loosely followed. Another reason might be the lack of knowledge or experience, which may 
make an individual deliberately avoid situations to share what they know in order to save face. 
Each of these reasons requires a different coaching process. 

Workgroups may benefit from providing more opportunities for members to interact - whether 
via web-based newsletter or web pages, face-to-face meetings, or frequent teleconferences. 
Workgroups may also benefit from understanding which of its members are central or core and 
which are considered peripheral and why. Workgroup leaders may need to reconstitute these sub- 
groups especially if it would increase accessibility to infonnation and knowledge. 
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Managing Knowledge 


Question: Mark the answer that represents your level of agreement with the statement provided: 


Managing Knowledge 

Category Description 

Frequency of Responses 
Over 3 

We frequently collaborate and share information within our workgroup 

110/133 = 83% 

Creative ideas are given support and recognition within our workgroup 

107/133 = 80% 

We produce quality deliverables 

107/133 = 80% 

Our workgroup knows how to apply what we leam toward improving what we 
do and how we do it 

96/132 = 73% 

We are able to solve complex problems effectively because we get the right 
people involved at the right time 

95/133 = 71% 

Our workgroup actively applies what we leam toward improving what we do 
and how do it 

94/133 = 70% 

We systematically document and distribute our knowledge 

80/133 = 60% 

We have mechanisms built into our workgroup processes to ensure that we 
leam from previous success and failures 

73/133 = 55% 


Overall Mean Average Score: 3.83 
Analysis 

The key purpose of collaborating is to produce something, whether it is solving a problem or 
creating something new. Over 80% of the respondents report that they frequently collaborate and 
share information within their workgroups. Respondents are adept at exploiting their 
collaborative synergy to solve complex problems and produce quality deliverables. This suggests 
that the majority of the participating workgroups value knowledge sharing and are accustomed to 
collaborating and that sharing knowledge is widely practiced and supported. 

While the results indicate that the survey respondents are apparently effective at collaborating 
and managing their knowledge, the results also show that more than of the workgroups are 
limited in their ability to learn from previous experiences. Of those who do harness key learning 
from their previous experiences, only 70% actively apply this learning toward process 
improvement. 

Furthermore, only 60% of the respondents indicated that they systematically document and 
distribute knowledge, which may be contributing to their inability to gain access to critical 
knowledge in a timely manner as indicated in the previous section. While these indications are 
not alarming, they do suggest that workgroups may benefit from revisiting their continuous 
improvement strategies and processes for gathering and distributing lessons learned in order to 
get the most out of the pilot. 
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Part III - Current Collaborative Environment: 
All Workgroups 


Virtual Team Meetings 


Question: Mark the answer that represents your level of agreement with the statement provided: 


Virtual Team Meetings 

Category Description 

Frequency of Responses 
Over 3 

Attending virtual meetings is a simple process 

95/134 = 71% 

Scheduling virtual meetings is a simple process 

63/131 =48% 

Communication in virtual meetings is adequate 

61/133 = 46% 

Sharing electronic information in virtual meetings is simple to do and 
effective 

56/132 = 42% 

Consensus is easily reached in virtual meetings 

40/133 = 30% 


Overall Mean Average Score: 3.35 
Analysis 

While over half of the respondents experience some degree of difficulty in scheduling virtual 
meetings, most respondents perceive their overall experience in attending virtual meetings as 
relatively simple. More than half of the respondents indicate that communicating and sharing 
electronic information in virtual meetings is less than adequate. Only 30% find group consensus 
easy to reach. 

Since WebEx is a tool designed specifically to facilitate scheduling, communicating, and sharing 
documentation during virtual meetings, workgroups should see marked improvements as they 
become more adept at using this tool throughout the pilot. 


NASA/TM— 2005-213210 


24 





Virtual Team Information Sharing 


Question: Mark the answer that represents your level of agreement with the statement provided: 


Virtual Team Information Sharing 

Category Description 

Frequency of Responses Over 3 

Team members are aware of their actions 

87/133 = 65% 

1 am aware of my team's schedule of events 

83/133 = 62% 

Actions items are effectively documented and tracked 

73/133 = 55% 

Sharing team information is simple and effective 

71/133 = 53% 

I can always find the current version of team documents 

56/133 = 42% 

Team leaders can easily track progress on actions 

53/133 = 40% 


Overall Mean Average Score: 3.43 
Analysis 

Nearly 40% of the respondents are not aware of their workgroup’s schedule of events and 
actions. Much worse, almost half of the respondents report that action items are not effectively 
documented and tracked. Respondents, in general, do not find information sharing simple. These 
findings are consistent with those in the Open Communication section of this report, which 
showed that nearly 40% of the respondents found access to information is difficult and complex. 
Clearly inaccessibility to basic and critical information is the most recurrent theme. 

In light of these findings, workgroups would do well to embrace eRoom as a mechanism to reach 
out and communicate with each other. Workgroup facilitators must look for ways to encourage 
members to use eRoom as a central hub for communicating between face-to-face meetings 
and/or teleconferences. Workgroup members must gain enough experience beyond the training 
exercises in order to realize the full value and benefits of the tool. 
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Productivity 


Question: How likely would you be to adopt for regular use software/services that: 


Productivity 

Category Description 

Frequency of Responses 
Over 3 

Allows you to enhance your virtual meetings by keeping discussions 
focused and actively engaging members 

63/73 = 86% 

Allows your virtual team members to simultaneously mark up a drawing or 
document on your desktop 

63/78 = 86% 

Allows you to make team documents and records available to the 
workgroup at all times 

54/67 = 80% 

Allows you to reduce or eliminate the need to travel to other destinations to 
make presentations 

59/83 = 71% 


**Responses indicate level of agreement 


Analysis 

Most of the respondents indicated that they would adopt collaborative software or services that 
enhance the richness of virtual meetings. Not only would the respondents appreciate software 
that would allow them to simultaneously modify or edit a document, but also archive that 
document and make it accessible to others at all times. These requirements overshadow the 
respondents need to adopt collaborative tools that would reduce the need for travel. 

Again, eRoom and WebEx are excellent tools that will allow pilot participants to enhance 
meeting activities and the flow of work products. 


NASA/TM— 2005-213210 


26 




Collaborative Tool Use 

Nearly 90% (119/135) of those that responded to this set of questions indicated that they were 
actively seeking new collaborative tools to improve productivity. The collaborative tools that 
workgroups reported using are: 


eRoom 

NetMeeting 

Livelink 

Docushare 

WebEx 

Telecons 

ViTS 

PBMA and other 
Intranet Websites 

* Other 

2 

20 

13 

3 

4 

7 

8 

7 

19 


*Other tools include: 

Government Off The Shelf (GOTS) product VSDE (1) 
Meeting Maker (1) 

Windchill (3) 

CUCMe (3) 

I-TIPS (1) 

Email (5) 

Smartboads (1) 

Productview (1) 

Science Organizer (1) 

Postdoc (1) 

PDS (1) 
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Part IV - Summary 


Respondents care about collaboration and are willing to work on improving their collaborative 
capabilities. Respondents are skilled in the four core collaborative competencies: trust, open 
communications, knowledge management, and motivation. Respondents have a high degree of 
trust in each other as competent professionals and their ability to perform in the best interest of 
the group. Respondents are motivated to do a good job and take pride in the outcomes and 
deliverables of the workgroup. 

While workgroup members practice and enjoy active listening and information exchange, many 
have difficulty accessing basic and critical project infonnation. Like most teams and groups, 
survey respondents do not have continuous improvement processes. 

Respondents realize that better access to project information would enhance their performance 
and collaborative capabilities as a team. In fact, the most recurrent theme in the analysis of 
responses was inaccessibility to project infonnation. Fortunately, the respondents are motivated 
to explore new collaborative technologies that enhance virtual meetings by keeping members 
actively engaged. For these reasons, participants will benefit from the tools being piloted in 
this study. 
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Appendix B 
Post-Pilot Survey 

Overall Pilot Questions 


1) Which Team Collaboration tool(s) did you attempt to use during this Pilot? 



Number of Responses 

% of Total 

eRoom and WebEx 

27 

21% 

WebEx only 

99 

79% 

Total 

126 

100% 


2) Which platform(s) did you use for the Team Collaboration tool(s)? 


Number of Responses % of Total 


Macintosh 21 17% 

PC/Windows 97 80% 

Linux 3 2% 

Unix/HP 1 1% 

(Note: Some responses had more than one system type indicated; 
some responses did not make a selection). 


eRoom Related Questions 

3) eRoom has enabled my team to get work done with less reliance on travel. 


Negative 

28% 

Neutral 

32% 

Positive 

40% 


4) eRoom has enabled collaboration among team members distributed across NASA Centers and external 
partners. 


Negative 4% 

Neutral 32% 

Positive 64% 


5) eRoom was easy to learn and use. 

Negative 12% 

Neutral 38% 

Positive 50% 


6) Considering the application of eRoom, check all that apply: 

Sharing team information is simple and effective. 88% 

I am aware of my team’s schedule of events. 38% 

Team members are aware of their action items. 25% 
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Action times are effectively documented and tracked. 1 9% 

Team leaders can easily track progress on actions. 19% 

7) Considering the capabilities you consider essential for a NASA Virtual Space service - please rate the 
importance of the following capabilities on a 1-5 scale where (1) is 'Not Important,' ... (5) is 'Very 
Important.' 

Rank 

1 Multi Platform Support 

2 Action Tracking 

3 Team Calendar 

4 Email Alerts/Notifications 

5 Coordination of document changes (Check In/Out) 

6 Threaded Discussions 

7 Databases 

8 Drag and Drop from Windows Desktop 

9 Polling/Voting 

8) If your team did not successfully adopt eRoom - please check your top 3, or so, primary reasons why. 

Rank 

N/A We did successfully adopt eRoom! (4 responses) 

1 Unclear (and perhaps undocumented) team organization and/or processes 

2 Did not want to take the risk of putting all the energy into shifting to this tool - and then not 

have it be adopted as a corporate solution - and have to transition all our data out 

3 Lack of commitment by the team and team leader to employ a new tool 

4 tie Unidentified 'pain points' that team is attempting to address with collaborative tools 

4 tie Lack of commitment by the team and team leader to alter work habits 

5 tie Unclear (or changing) team goals and deliverables 

5 tie Lack of tool training 

5 tie Tool did not do what I needed it to do - it did not deliver on needed functionality 

6 tie Tool too difficult 

6 tie Already have this functionality - so don't want to switch from the tool we use now 

7 tie Lack of collaborative tool facilitation assistance from outside the team 

7 tie Lack of collaborative tool adoption assistance (internal resources) within the team 

7 tie Duration of the pilot not long enough to get started 

8 Lack of executive sponsorship (support) 

9) Overall, our team successfully used eRoom. 


Negative 

30% 

Neutral 

43% 

Positive 

26% 
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WebEx Meeting Center Related Questions 

10) WebEx Meeting Center has enabled my team to get work done with less reliance on travel. 


Negative 

9% 

Neutral 

18% 

Positive 

73% 


1 1) WebEx Meeting Center has enabled collaboration among team members distributed across NASA 
Centers and external partners. 


Negative 5% 

Neutral 13% 

Positive 82% 

12) WebEx Meeting Center was easy to learn and use. 

Negative 4% 

Neutral 17% 

Positive 79% 


13) Considering WebEx Meeting Center to conduct virtual meetings, rate all that apply: 

Rank 

1 Attending virtual meetings is a simple process. 

2 Attendees are able to participate effectively in virtual meetings. 

3 Sharing electronic information in virtual meetings is simple to do and effective. 

4 Scheduling virtual meetings is a simple process. 

5 Consensus is easily reached in virtual meetings. 

14) Considering the capabilities you consider essential for a NASA Virtual Meeting service - please rate 
the importance of the following capabilities on a 1-5 scale where (1) is 'Not Important,' ... (5) is 'Very 
Important.' 

Rank 

1 Presentation Sharing 

2 Simple attendance by external partners 

3 Application Sharing 

4 Multi platform support 

5 Integration with telephone based voice conferencing 

6 Desktop Sharing 

7 Meeting recording and playback 

8 tie Polling 

8 tie Video 

9 VoIP 

15) Overall, our team successfully used WebEx Meeting Center. 


Negative 

7% 

Neutral 

14% 

Positive 

79% 
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General Pilot Questions 


16) Our team has a clear mission, schedule, and deliverables. 

Negative 10% 

Neutral 16% 

Positive 74% 

17) My team embraced the Team Collaboration Tool(s) offered by the pilot. 

Negative 8% 

Neutral 24% 

Positive 68% 

18) Considering the ingredients you consider essential to successful team adoption of collaboration tools - 
please rate the importance of the following ingredients on a 1-5 scale where (1) is 'Not Important,' ... (5) is 
'Very Important.' 

Rank 

1 Easy to use tools 

2 Commitment by the team and team leader to alter work habits and employ new tools 

3 Coiporate commitment to tool support and availability 

4 Executive Support 

5 Collaborative tool adoption assistance from inside the team 

6 Clear team goals and deliverables 

7 Clear (and perhaps documented) team processes 

8 Collaborative tool facilitation assistance from outside the team 

9 Clear team vision/mission statement 

1 0 Agreed to “pain points” (most pressing problems) that team is attempting to address with 

collaborative tools 

1 1 Critical mass of NASA people already using tools 

19) Please list any challenges that made adoption of the pilot’s collaborative tools difficult, check all that 
apply: 


Rank 

1 N/A 

2 My team members were not able to commit enough time to learn features of the 

application 

3 My team relied on, and only required, traditional/basic forms of communication - 

phone, email, fax 

4 The leadership of my team changed 

5 My team was reorganized 

6 Our project lost funding 

7 My team members do not see the value of using collaborative tools 

20) Please rate the Support that you received during the Team Collaboration pilot. 


Negative 

2% 

Neutral 

30% 

Positive 

76% 
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21) A standard set of tools for team spaces and virtual team meetings (e.g., eRoom and WebEx) would 
benefit NASA. 


Negative 

2% 

Neutral 

12% 

Positive 

86% 


22) My team/project/program/Center/enterprise would likely be willing to pay for these types of 
capabilities. 

Yes 
No 


48% 

52% 
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Appendix C 
Acronyms 


ASP application service provider 

BRT Business and Restricted Technology Information 

CIO Chief Information Officer 

EAR Export Administration Regulations 

eGOV electronic Government (initiative) 

FAQ frequently asked questions 
GOTS Government off-the-shelf (software) 

IT information technology 

ITS Information Technology Security 

ITAR International Traffic in Arm s 

NPG NASA Procedures and Guidelines 

POC point of contact 

PSRP Payload Safety Review Panel 

SSC support service contractor 

SSL secure sockets layer 

TC team coordinator 

TF team facilitator 

TL team leader 
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